1. Field of the Invention
The present invention relates to an image processing method and a device for a driver assistance system of a motor vehicle for detecting and classifying at least one portion of at least one predefined image element, having a road sign or portions of a road sign in at least one digital image which is to be searched and which is captured by an image sensor of the driver assistance system.
2. Description of Related Art
A method is known from published German patent application document DE 198 42 176 A1 for recognizing road signs in the surroundings of a vehicle and for navigating the vehicle, in which road sign recognition data are generated when a road sign is recognized.
Unlike in countries which have joined the Vienna Convention, in particular for regulating the design of road signs, in countries such as the United States, for example, road signs may be very different and have different shapes. In addition, in the United States the individual states have the responsibility for the design of road signs, as the result of which variations with respect to size, style of writing, configuration, etc., may occur. Furthermore, instead of pictograms as used under the Vienna Convention, text such as “SPEED LIMIT,” “TRUCKS,” “MINIMUM SPEED,” or the like is used. This represents a particular challenge for a driver assistance system based on the recognition of road signs.
In countries in which the road signs are designed in accordance with the Vienna Convention, the driver assistance system is typically able to initially detect the shape, for example a circle for speed limits, in an image which is captured in particular by an image sensor of the driver assistance system. This is likewise possible for U.S. road signs (for example, a rectangular sign). The image details are subsequently normalized with respect to their brightness in order to minimize influences of the lighting situation. In addition, the image details are normalized with respect to their size on the stored pictograms of the road signs to be classified. Lastly, the pictograms are compared by grayscale comparisons with the image details, and when there is an adequate match the image detail is recognized as a road sign. In support, motion information may also be taken into account in order to distinguish, for example, road signs affixed at the rear area of trucks or buses from genuine stationary road signs. The above-described procedure has only limited usability for U.S. road signs in the United States, since the mentioned numerous variations in road signs greatly increase the number of pictograms to be stored, provided that they may even all be found in advance, and thus increase the computing complexity.
An alternative option would be to read, so to speak, the individual characters, i.e., letters and numbers, in the detected road sign, and to interpret them in the sense of optical character recognition (OCR). However, this is relatively complicated and difficult to implement in motor vehicle control units. In addition, the general understanding of the written words is not necessary in principle, since only a limited number of key words are involved.
Methods in image processing have been developed recently which are able to invariantly describe image regions with respect to scaling and rotation. Scale-invariant feature transform (SIFT) is an algorithm for extracting local image features from images, and is used primarily in image recognition. U.S. Pat. No. 6,711,293 B1 discloses a method and a device for identifying scale-invariant features in an image, as well as a method and a device for using such scale-invariant features for locating an object in an image.
Furthermore, an algorithm for quick and robust ascertainment of image features for computer vision is known from Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, “SURF: Speeded-up robust features,” Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346-359, 2008.
With regard to further related art, reference is made to published German patent application document DE 103 38 455 A1.